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(54) MOVING PICTURE EXPANDING METHOD AND RECORDING MEDIUM 
RECORDING PROGRAM EXECUTING THE SAME 

(57)Abstract: 

PROBLEM TO BE SOLVED^ To suppress cache errors in the movement compensation 
of a B picture by alternately repeating expansion of one macro block of a first frame and 
that of a second frame and simultaneously executing expansion of the two frames when 
the two adjacent frames are the B pictures. 

SOLUTION- When two adjacent firames are B pictures, expansion of one macro block of 
the first frame and that of the second frame are alternately repeated and the expansion 
processings of the two frames are simultaneously executed. In the moving picture 
expansion device, a bit stream compressed by MPEG is read from an outer storage 
device 304 and is temporarily stored in a memory 302. The stored bit stream is 
expanded by the operation of decoder software run on CPU 301 and picture data is 
stored in the memory 302 as the result. A program for expansion processing is stored on 



floppy disk 307. 
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JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1 This document has been translated by computer. So the translation may not 
reflect the original precisely. 



2.**** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 

CLAIMS 
[Claim(s)] 

[Claim 1] It is the dynamic-image expanding approach performed by the 
dynamic-image expanding equipment constituted including CPU and memory 
which built in cache memory. It is the dynamic-image expanding approach 
performed by the dynamic-image expanding equipment driven with the software 
for dynamic-image expanding, adjoining - plurality — B — a picture — expanding 
~ processing ~ carrying out ~ the time ~ these — adjoining ~ B — a picture ~ 
each — ******** — corresponding — a location ~ it is — a constant — (— K ~) ~ an 
individual a macro - a block - receiving ~ expanding - processing - 
alternation ~ performing ~ things — the description — ** ~ having carried out — a 
dynamic image ~ expanding - an approach . 

[Claim 2] It is the dynamic-image expanding approach characterized by said K 
being 1 in the dynamic-image expanding approach according to claim 1. 
[Claim 3] cache memory - having built - CPU ~ memory - containing ~ 



constituting — having had — a dynamic image - expanding - equipment -- 
operating - making — a sake — a program — having recorded — a record medium 
— it is - adjoining - plurality — B - a picture - expanding - processing — 
carrying out ~ the time - these ~ every - B - pictures ~ each ~ ******** « 
corresponding - a location — it is - a constant ~ (- K -) - an individual ~ a 
macro — a block — receiving expanding — processing -- alternation — 
performing — a step — containing — a program — having recorded — a record 
medium . 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] This invention relates to the technique for accessing 
cache memory efficiently, when performing expanding processing of 
dynamic-image data especially by software about the technique which elongates 
the compressed dynamic-image data. 
[0002] 

[Description of the Prior Art] The international-standards method called MPEG 
(Moving Picture coding Experts Group) as the compression for storing quality 
dynamic-image data or communicating and an expanding method is defined. 
Since computational complexity huge in order to elongate the dynamic-image 
data compressed based on the specification of MPEG on real time was needed, 
the hardware designed specially was used conventionally. However, high 



performance-ization of the latest microprocessor is enabling it to realize 
expanding of the dynamic-image data compressed by the usual microprocessor 
only by software. 

[0003] Compression and the expanding algorithm of MPEG are stated to "April, 
1991. communique SHONZU OBU THE ESHIEMU, the 34th volume, and No. 4 
(Communications of the ACM, Vol.34, No.4, April, 1991)" in detail, for example. 
The expanding algorithm of MPEG consists of decode processing of a 
variable-length sign, reverse quantization processing, reverse discrete cosine 
transform processing, and motion compensation processing, if it says roughly. 
[0004] With reference to drawing 3 , the example constituted from combination of 
the system centering on CPU301 as shows MPEG expanding equipment to 
drawing 3 , and the software on CPU301 is explained. 

[0005] The bit stream compressed by MPEG is stored in external storage 304. A 
bit stream is read from the external storage 304, such as CD-ROM, and is 
temporarily stored in memory 302. The bit stream stored in memory 302 is 
elongated by the work of decoder software which operates on CPU301, and 
image data is stored in memory 302 as the result. The image data stored in 
memory 302 is changed into the format for a display by the software on CPU, 



and is transmitted to a display 303. Decoder software is stored in the external 
storage of floppy disk drive unit (FD) 307 grade. 

[0006] Drawing 4 shows the sequence of an image frame at the time of 
performing compression and expanding by MPEG. MPEG has three kinds of 
picture types called I picture, P picture, and B picture as a picture type for 
encoding the image of one frame. I picture is a picture type compressed only 
using the information in the frame, without performing motion compensation 
processing. P picture is a picture type with which motion compensation 
processing in which the frame is predicted is performed from the image 
information of the past frame rather than the frame. B picture is a picture type 
with which motion compensation processing in which bidirectional prediction of 
the picture signal of the frame is carried out is performed from the image 
information of both the past frame and the frame of the future rather than the 
frame. 

[0007] Drawing 4 (A) shows the assignment of a typical picture type at the time 
of compressing a dynamic image by MPEG. Frame #1 is I picture and it is 
compressed, without referring to the image information of other frames. Frame 
#2 are B picture and they are compressed by the bidirectional motion 



compensation processing using frame #1 and frame #4. Frame #3 are B picture 
and they are compressed by bidirectional motion compensation processing with 
reference to frame #1 and frame #4. Frame #4 are P picture and they are 
compressed by the motion compensation processing using frame #1 . Frame #5 
are B picture and they are compressed by the bidirectional motion compensation 
processing using frame #4 and frame #7. Frame #6 are B picture and they are 
compressed by the bidirectional motion compensation processing using frame 
#4 and frame #7. Frame #7 are P picture and they are compressed by the 
motion compensation processing which referred to frame #4. The arrow head of 
a dotted line expresses inter-frame reference relation. 

[0008] Drawing 4 (B) shows the processing sequence of each frame in the case 
of performing compression and expanding according to the picture type 
described in the top. Before performing compression processing, an input 
dynamic-image signal is rearranged like drawing 4 (A) to drawing 4 (B). 
[0009] In MPEG, frame #1 which is I picture first is compressed. Next, frame #4 
which are P picture are compressed with reference to the local decode image of 
frame #1 . And frame #2 which are B picture are compressed with reference to 
the local decode image of frame #1 and frame #4. Moreover, frame #3 which are 



B picture are compressed with reference to the local decode image of frame #1 
and frame #4. Frame #7 which are P picture are compressed with reference to 
the local decode image of frame #4. Then, frame #5 are compressed with 
reference to the local decode image of frame #4 and frame #7. Frame #6 are 
compressed with reference to the local decode image of frame #4 and frame #7. 
[0010] The information on each frame is described by the sequence which 
compressed the frame into the MPEG bit stream which describes the 
compressed image. That is, the sequence of the frame inputted into an expander 
differs from the sequence of the frame of a dynamic image from the first. 
[0011] Thus, in case the compressed MPEG bit stream is elongated, frame #1 is 
elongated first. And frame #4 (P picture) is elongated with reference to the image 
of elongated frame #1. Frame #2 (B picture) is elongated with reference to the 
image of frame #1 elongated, frame #4 which were elongated, and 2 **. Frame 
#3 (B picture) is elongated with reference to two images of frame #1 elongated 
and frame #4 which were elongated. Frame #7 (P picture) is elongated with 
reference to the image of elongated frame #4. Then, frame #5 are elongated 
with reference to two images of frame #4 elongated and frame #7 which were 
elongated. Frame #6 are elongated with reference to two images of frame #4 



elongated and frame #7 which were elongated. Drawing 4 (C) shows the 
sequence for displaying the elongated image on the display 303 grade of 
drawing 3 . 

[0012] Actuation by the program stored in decoder software, i.e., the floppy disk 
drive unit of drawing 3 , is explained with reference to the flow chart of drawing 5 . 
[0013] If the step of variable-length sign decode (502), reverse quantization 
(503), reverse DCT (504), and a motion compensation (505) is repeated for 
every 1 macro block and expanding processing of one frame processes the 
macro block of the last of one frame, expanding processing of one frame will be 
ended (507). In addition, motion compensation processing (505) is not 
performed to I picture. Moreover, in motion compensation processing (505), to P 
picture, uni-directional motion compensation prediction is performed and 
bidirectional motion compensation prediction is performed to B picture. 
[0014] The variable-length sign decode (502) for 1 macro block reads the bit 
stream stored in memory 302, and decodes Huffman coding by referring to a 
several bits variable-length sign table at a time. At this time, the location where 
the pointer (POINTER) which points out the location on memory read the bit 
stream is memorized. When the bit stream was read, only the always read 



amount was advanced, and this pointer has pointed out the location of the bit 
stream which should always be read to a degree. The variable-jength sign table 
used here is beforehand stored in memory 307 with decoder software, and in 
case it performs variable-length sign decode processing, it is read in CPU. The 
decode result of Huffman coding is stored in one field on the memory 302 called 
a DCT multiplier buffer. 

[0015] Reverse quantization (503) is processing which asks for the product of 
the value stored in the DCT multiplier buffer, and the quantization parameter 
contained in the MPEG bit stream. The result of reverse quantization is returned 
to the above-mentioned DCT multiplier buffer. 

[0016] Reverse DCT (504) gives a two-dimensional reverse discrete cosine 
transform to the reverse-quantized value which is stored in the DCT multiplier 
buffer. The result of a two-dimensional reverse discrete cosine transform is 
returned to the same DCT multiplier buffer. 

[0017] A motion compensation (505) is a thing using the time redundancy of a 
dynamic image, and is processing which cuts down some images from the 
already elongated frame, and restores an image. The location of the image 
[ finishing / expanding / already ] cut down is determined by the parameter (this 



parameter is also contained in the IVIPEG bit stream) called a motion vector. The. 
image cut down and restored is a field of the square called a macro block which 
consists of 16 pixels long and 16 pixels wide. Expanding of a macro block is 
completed by being added with the result of reverse DCT by which the started 
field is stored in the DCT multiplier buffer. The elongated macro block is stored in 
memory 302. 

[0018] In the system as shown in drawing 3 , cache memory 305 is usually built 
in CPU301 . Although capacity is small compared with cache memory in memory, 
it is the memory which can access a high speed. The contents of the external 
memory 302 which CPU301 accessed recently are stored in cache memory. 
After accessing, the contents without long duration access are controlled to 
****** from cache memory and to be returned to memory 302. 
[0019] That is, since the contents remain in the cache, access of the contents 
which CPU accessed once is possible, as long as it accesses immediately after 
that again within the machine cycle of CPU. This is called a cache hit. However, 
when having not accessed once yet, or in [ since long duration access was not 
carried out once it accessed, ] accessing the contents of the memory which is 
not stored in cache memory, it takes the dozens times [ several to ] as many time 



amount of the machine cycle of CPU as this. This phenomenon is called a cache 
mistake. 

[0020] While waiting for access of data by cache mistake, actuation of the whole 
processor will, stop at a microprocessor. Therefore, in order to raise the 
processing speed of software, it is important to make into smallness the 
probability for a cache mistake to occur, as much as possible, and to enable it to 
use the capacity of CPU effectively. 

[0021] In the microprocessor in this time, the capacity of the cache memory built 
in is 10 K bytes in maximum number. Mass cache memory may be arranged 
rather than the cache memory which is called secondary cache memory to the 
outside of CPU301 depending on the structure of a system and which is built in. 
At present, the capacity of secondary cache memory is 100 K bytes in maximum 
number. On the other hand, the capacity of one frame of the image treated by 
MPEG is considered by the screen size from dozens of K bytes to several 
megabytes. In the case of the specification which is called MPEG 2 Maine profile 
Maine level and which is used present the general-purpose, since one frame 
becomes hundreds of K bytes, only the image of one frame is storable in a 
secondary cache at the maximum. 



[0022] The behavior of cache memory when elongating MPEG 2 IVIaine profile 
Maine level with the expanding equipment of a Prior art is explained. For 
example, a series of actuation which elongates frame #2 of drawing 4 (B) and 
then elongates frame #3 is explained. 

[0023] In expanding processing of frame #2, in order to refer to frame #1 and 
frame #4, it is necessary to access the image of frame #2 elongated from now on, 
and the image for three frames in all by motion compensation processing, the 
phase which expanding processing of frame #2 ended since the image for three 
frames was unstorable in cache memory — cache memory — each of frame #1 , 
frame #4, and frame #2 -- only the part is stored. In order that the expanding 
algorithm of MPEG may advance processing caudad toward the right from Hidari 
of a screen, in many cases, the elongated image of the frame referred to is also 
accessed in order towards a lower limit from the upper limit on a screen. 
Therefore, the data of the part near the upper limit of a screen are driven out of 
cache memory as processing of expanding progresses, and finally only the data 
of the part near a lower limit will remain in cache memory. 
[0024] In motion compensation processing of the processing which elongates 
frame #3 succeeding ly, frame #1 and frame #4 are referred to again. While 



elongating the part near the screen upper limit of frame #3, the part near the 
screen upper limit of frame #1 and frame #4 will be referred to, but since only the 
data of the part near the lower limit of frame #1 and frame #4 remain in cache 
memory as a result of expanding processing of frame #2, all accesses cause a 
cache mistake. Moreover, since the data of the part near [ in the phase where 
expanding processing of the part near upper limit is performed ] the lower limit of 
frame #1 and frame #4 which remained in cache memory are driven out, all 
accesses will cause a cache mistake also for the processing which elongates the 
part near the screen lower limit of frame #3. 

[0025] Thus, with the dynamic-image expanding equipment of a Prior art, since 
the whole frame referred to in motion compensation processing is unstorable in 
cache memory, cache mistakes occur frequently. When two or more frames 
which adjoin especially are B pictures, in order to have to read a cache mistake 
for the same reference frame with a lifting repeatedly, there is much futility. 
Therefore, with this conventional technique, cache mistakes occur frequently 
and it has become the factor of a rate fall of decoding software. 
[0026] 

[Problem(s) to be Solved by the Invention] Thus, since cache memory in CPU 



cannot be effectively used in expanding processing of B picture especially with 
the conventional dynamic-image expanding equipment built by software, 
processing speed is slow. The latency time for memory access is the big factor 
of a rate fall of decoder software. This invention controls the cache mistake in 
motion compensation processing of B picture, and aims at offering the 
dynamic-image expanding technique in which the rate fall of decoding software 
was prevented. 
[0027] 

[Means for Solving the Problem] When performing expanding processing of a 
certain frame, it is B picture, and the 1st dynamic-image expanding equipment of 
this invention is repeating the expanding processing for 1 macro block of the 1st 
frame, and the expanding processing for 1 macro block of the 2nd frame by turns, 
if the following frame's is also B picture, and performs adjoining expanding 
processing of two frames to coincidence. When performing expanding 
processing of a certain frame, it is B picture, and the 2nd dynamic-image 
expanding equipment of this Invention is repeating the expanding processing for 
a fixed number of macroes block with the 1st frame, and the expanding 
processing for a fixed number of macroes block with the 2nd frame by turns, if 



the following frame's is also B picture, and performs adjoining expanding 
processing of two frames to coincidence. 

[0028] In this invention, when both two adjoining frames are B picture, in order to 
perform the expanding processing of two frames by turns per one or more macro 
blocks, expanding processing of the macro block in the location where the 
adjoining frame is the same will be performed continuously: Thereby, the 
probability for a cache mistake to occur in motion compensation processing can 
be made low. The motion compensation processing in the macro block which 
this has in the location where the adjoining frame is the same is based on the 
property to use the field in the near location in a reference frame in many cases. 
[0029] 

[Embodiment of the Invention] Only the programs in which this invention and the 
conventional technique were stored in external storage 307 differ so that it may 
become clear from the following explanation. The bit stream compressed by 
MPEG is stored in external storage 304. A bit stream is read from external 
storage 304, and is temporarily stored in memory 302. The bit stream stored in 
memory 302 is elongated by the work of decoder software which operates on 
CPU301, and image data is stored in memory 302 as the result. The program 



explained below using the flow chart of drawing 1 or drawing 2 is stored in the 
floppy disk 307. 

[0030] Actuation of the 1st operation gestalt of this invention is explained with 
reference to drawing. Drawing 1 Is a flow chart which shows actuation of the 1st 
operation gestalt of this invention. 

[0031] In expanding processing of one frame, it is judged first whether the 
picture type of the frame is B picture (102). 

[0032] If the inputted picture type of a frame is I picture or P picture, like a Prior 
art, the expanding processing for 1 macro block (103; the same as that of 
processing of steps 502-505 of drawing 5 ) is repeated (103,107), and if the 
macro block of the last of one frame is processed, expanding processing of one 
frame will be ended (108). 

[0033] The frame will be called Frame X when [ in step 102 ] a picture type is B 
picture as a result of a judgment. In that case, the pointer (POINTER) which has 
memorized the current bit stream read-out location is saved at the variable pi 
(109), and a bit stream is skipped to the head of the following frame (110). 
[0034] This following frame will be called Frame Y. The bit stream readout 
location which saves the bit stream read-out location at that time at the variable 



p2, and has been saved p1 is returned to a pointer (111). 

[0035] If the picture type of Frame Y is not B picture (112), expanding processing 
of Frame X will be performed like I picture or P picture. 

[0036] If the picture type of Frame Y is also B picture, expanding processing for 
1 macro block of Frame X (1 13; the same as that of processirig of steps 502-505 
of drawing 5 ) will be performed first. 

[0037] Next, the pointer which points out the head of the frame Y which saves a 
pointer at a variable pi arid is saved p2 is returned to a pointer (117), and 
expanding processing for 1 macro block of Frame Y (118; the same as that of 
processing of steps 502-505 of drawing 5 ) is performed. 

[0038] Next, the bit stream readout location which saves a pointer at a variable 
p2 and is saved pi is returned to a pointer (122), and expanding processing for 1 
macro block of Frame X (1 13; the same as that of processing of steps 502-505 
of drawing 5 ) is performed. 

[0039] If expanding processing of Frame X and Frame Y is repeated by turns for 
every 1 macro block and the last macro block is processed like the following The 
expanding processing for two frames of Frame X and Frame Y is ended (124). 
[0040] Next, actuation of the 2nd operation gestalt of this invention is explained 



with reference to drawing. 

[0041] Drawing 2 is a flow chart which shows actuation of the 2nd operation 
gestalt of this invention. Let K be a certain constant. In expanding processing of 
one frame, it judges first whether the picture type of the frame is B picture (202). 
If the picture type of the frame is I picture or P picture, the expanding processing 
for 1 macro block (203; the same as that of processing of steps 502-505 of 
drawing 5 ) is repeated like a Prior art (203,207), and if the macro block of the 
last of one frame is processed, expanding processing of one frame will be ended 

(208) . 

[0042] The frame will be called Frame X when [ in step 202 ] a picture type is B 
picture as a result of a judgment. In that case, the pointer pointer which has 
memorized the current bit stream read-out location is saved at the variable p1 

(209) , and a bit stream is skipped to the head of the following frame (210). 
[0043] This following frame will be called Frame Y. The bit stream read-out 
location at that time is saved at the variable p2, and the bit stream read-out 
location saved pi is returned to a pointer (21 1). 

[0044] If the picture type of Frame Y is not B picture (212), expanding processing 
of Frame X will be performed like I picture or P picture. 



[0045] If the picture type of Frame Y is also B picture, expanding processing for 
K macro blocks of Frame X (213, 214, 218, 219) will be performed first. Next, the 
pointer which points out the head of the frame Y which saves a pointer at a 
variable pi and is saved p2 is returned to a pointer (220), and expanding 
processing for K macro blocks of Frame Y (221 , 222, 226, 227) is performed. 
[0046] Next, the bit stream read-out location which saves a pointer at a variable 
p2 and is saved p1 is returned to a pointer (228), and expanding processing for 1 
macro block of Frame X (213, 214, 218, 219) Is performed. 
[0047] If expanding processing of Frame X and Frame Y is repeated by turns for 
every K macro blocks and the last macro block is processed like the following, 
the expanding processing for two frames of Frame X and Frame Y will be ended 
(230). 

[0048] Here, the optimal value changes with sizes of the image which K 
elongates with the size of cache memory. With for example, the dynarhic-image 
expanding equipment containing the cache memory which is 256 K bytes It Is 
referred to as 45 which is the number of a macro block of K of width 1 train on a 
screen when elongating the image of the MPEG 2 Maine profile Maine level of 
720 pixels wide and 480 pixels long. 



[0049] In addition, when both two frames that adjoins here were B picture, the 
example which performs the expanding processing of two frames by turns was 
described, but when all of three or more adjoining frames are B pictures, it is 
also possible to perform the expanding processing of three or more frames by 
turns. 
[0050] 

[Effect of the Invention] The effectiveness of this invention is preventing the rate 
fall of decoding software by reducing the cache mistake in motion compensation 
processing of B picture. With the gestalt of operation of this invention, when both 
two adjoining frames are B picture, expanding processing of one or more macro 
blocks in the location where the adjoining frame is the same will be performed 
continuously. This is enabled to use effectively the data of a reference frame 
read into a cache in motion compensation processing, and the probability for a 
cache mistake to occur can be made low. 
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[Brief Description of the Drawings] 

[Drawing 1] It is the flow chart which shows actuation of the gestalt of operation 
of the 1 St of this invention. 

[Drawing 2] It is the flow chart which shows actuation of the gestalt of operation 
of the 2nd of this invention. 

[Drawing 3] It is the block diagram showing the configuration of the gestalt of 
operation of this invention. 



[Drawing 4] It is drawing showing the processing sequence of the frame in 
dynamic-image expanding processing. 

[Drawing 5] It is the flow chart which shows actuation of a Prior art. 
[Description of Notations] 

301 CPU 

302 Memory 

303 Display 

304 External Storage 

305 Cache Memory 

306 Bus 

307 Floppy Disk Drive Unit 



